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DETAILED ACTION 
Double Patenting 

1 . The nonstatutory double patenting rejection is based on a judicially created doctrine 
grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or 
improper timewise extension of the "right to exclude" granted by a patent and to prevent possible 
harassment by multiple assignees. A nonstatutory obviousness-type double patenting rejection 
is appropriate where the conflicting claims are not identical, but at least one examined 
application claim is not patentably distinct from the reference claim(s) because the examined 
application claim is either anticipated by, or would have been obvious over, the reference 
claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re 
Goodman, 1 1 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re LongU 759 F.2d 887, 225 
USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re 
Vogel 422 F.2d 438, 164 USPQ 619 (CCPA 1970); and In re Thorington, 418 F.2d 528, 163 
USPQ 644 (CCPA 1969). 

A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may 
be used to overcome an actual or provisional rejection based on a nonstatutory double patenting 
ground provided the conflicting application or patent either is shown to be commonly owned 
with this application, or claims an invention made as a result of activities undertaken within the 
scope of a joint research agreement. 
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Effective January 1, 1994, a registered attorney or agent of record may sign a terminal 
disclaimer. A terminal disclaimer signed by the assignee must fully comply with 37 CFR 
3.73(b). 

2. . Claims 1-25 are provisionally rejected on the ground of nonstatutory obviousness-type 
double patenting as being unpatentable over claims 1-16 and 48-49 of copending Application 
No. 10/617,422. Although the conflicting claims are not identical, they are not patentably 
distinct from each other because both claimed inventions are directed to the same system/method 
for executing a multimodal software application utilizing input and output peripheral devices, 
workflow descriptions based on the software application, and interface components. Although 
App. No. 10/617,422 omits the prompt control of the presently claimed invention, the omission 
of such an element would be obvious to one of ordinary skill in the art because both systems 
essentially perform the same steps of providing input data, processing that data according to a 
workflow, and providing output data (i.e., a prompt) in response. 

This is a provisional obviousness-type double patenting rejection because the conflicting 
claims have not in fact been patented. 

Claim Objections 



3. Claims 7, 10, and 22-25 are objected to because of the following informalities: 

In Claim 7, line 2, "voice recognition system" should be changed to -speech recognition 
system- in order to be consistent with the terminology used in the specification and claim 12. 
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In Claim 10, Line 2 and claim 22, line 7, the acronym "GUI" should be expanded upon 
(i.e., graphical user interface) in order to clarify its meaning in the claim. 

Dependent claims 23-25 further limit an objected claim, and thus, are also object to due 
to minor informalities. 

Appropriate correction is required. 

Claim Rejections - 35 USC § fl2 

4. The following is a quotation of the first paragraph of 35 U.S.C. 1 12: 

The specification shall contain a written description of the invention, and of the manner and process of making 
and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it 
pertains, or with which it is most nearly connected, to make and use the same and shall set forth the best mode 
contemplated by the inventor of carrying out his invention. 

5. Claims 9-25 are rejected under 35 U.S.C. 1 12, first paragraph, as failing to comply with 
the written description requirement. The claim(s) contains subject matter which was not 
described in the specification in such a way as to reasonably convey to one skilled in the relevant 
art that the inventor(s), at the time the application was filed, had possession of the claimed 
invention. More specifically, claims 9 and 22 recite a plurality of peripheral devices capable of 
outputting a prompt and controlling the prompt according to an input state of the same devices, 
which is not disclosed by the specification. Instead, the output prompt appears to be controlled 
based not an on an input state of output devices, but on those states of a different plurality of 
peripheral devices (i.e., multimodal input devices) (see specification, page 27 and the third 
limitation of claim 1). 
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The dependent claims 10-21 and 23-25 further limit rejected independent, and thus, are 
also directed to non-statutory subject matter. 

Claim Rejections - 35 USC § 101 

6. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or 
any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and 
requirements of this title. 

7. Claims 1-25 are rejected under 35 U.S.C. 101 because the claimed invention is directed 
to non-statutory subject matter. 

Claims 1, 9, and 22 are drawn to a system, method, and program for executing a 
multimodal software application featuring a multimodal software application, dialog engine, and 
an interface component. In order for a claimed invention to be considered statutory under 35 
U.S.C. 101, it must be useful and accomplish a practical application. That is, it must produce a 
"useful, concrete and tangible result" (State Street, 149 F.3d at >1373-74<, 47 USPQ2d at 
1601-02), In this case, the final result is only an abstract prompt control signal generated within 
a computer system and not a tangible real-world output (for example, a display that displays the 
generated prompt or a speaker for playing out an audio prompt, Specification, Page 12). 
Moreover, even if such a tangible result were recited in the claimed invention, claim 1 would still 
be directed to non-statutory subject matter because the claimed dialog engine is not required to 
produce (i.e., does not actively produce) a tangible result since it is merely "configured" to do so. 

Claim 22 is drawn to program "instructions" as recited in the preamble and as such is 
non-statutory subject matter. Furthermore, the claimed medium is not limited to a tangible 
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computer readable medium (see Specification, Page 10), See MPEP § 2106.F/.B.l.a. Data 
structures not claimed as encoded in computer readable media are descriptive material per se and 
are not statutory because they are not capable of causing functional change in the computer. See, 
e.g., Warmerdam, 33 F.3d at 1361, 31 USPQ2d at 1760 (claim to a data structure per se held 
nonstatutory). Such claimed data structures do not define any structural and functional 
interrelationships between the data structure and other claimed aspects of the invention, which 
permit the data structure's functionality to be realized. In contrast, a claimed tangible computer 
readable medium encoded with a data structure defines structural and functional 
interrelationships between the data structure and the computer software and hardware 
components which permit the data structure's functionality to be realized, and is thus statutory. 
Similarly, computer programs claimed as computer listings per se, i.e., the descriptions or 
expressions of the programs are not physical "things." They are neither computer components 
nor statutory processes, as they are not "acts" being performed. Such claimed computer 
programs do not define any structural and functional interrelationships between the computer 
program and other claimed elements of a computer, which permit the computer program's 
functionality to be realized. 

Although the system in claim 1 is directed to a seemingly patentable system for dialog 
execution, this claim is directed towards non-functional descriptive material (i.e., computer 
program elements) as evidenced by Pages 9-10 of the specification. The specification indicates 
that the currently claimed system is drawn to program routine elements. In claim 1, these 
program elements are not stored on a tangible computer readable medium encoded with a 
program that enables the program's functionality to be realized when executed by a computer or 
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claimed in combination with other hardware elements that enable the program's functionality to 
be realized (i.e., display, speaker, speech recognizer, touch screen, etc). Thus, claim 1 is 
directed to non- statutory subject matter. 

Although the process in claim 9 is directed to a seemingly patentable process for dialog 
execution, this claim is directed towards non-functional descriptive material (i.e., computer 
program description) as evidenced by Claim 22. Claim 22 indicates that these steps are part of a 
computer program. In claim 9, this program description is not stored on a tangible computer 
readable medium encoded with a program that enables the program's functionality to be realized 
when executed by a computer, and thus, claim 9 is directed to non-statutory subject matter, for 
the same reasons as claim 22. 

The dependent claims 2-8, 10-21, and 23-25 further limit rejected independent, and thus, 
are also directed to non-statutory subject matter. 

Claim Rejections - 35 USC §102 

8. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 

9. Claims 1-3, 7-9, 11-13, 15, and 19-20 are rejected under 35 U.S.C. 102(b) as being 
anticipated by Morin et al (U.S. Patent: 5, 748,841). 

With respect to Claim 1, Morin discloses: 
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The multimodal software application, wherein said multimodal software application is 
configured to receive first data input from a first set of peripheral devices and output second data 
to a second set of peripheral devices (multimodal target software application capable of 
receiving data from a set of input devices and outputting data from a set of output devices. Col 
1 3, Lines 30-67; and various input and output devices, Col 11, Line 40- Col 12, Line 15)\ 

A dialog engine in communication with the multimodal software application, wherein 
said dialog engine is configured to execute a workflow description received from the multimodal 
software application and provide. the first data to the multimodal software application (dialog 
manager that executes dialog tasks in accordance with an application script and provides input 
data to an application, Col 9, Line 1- Col 10, Line 25); 

Said dialog engine further configured to control outputting of a prompt from the 
workflow description based on an input state of the first set of peripheral devices (dialog 
manager utilizing context and history to monitor an input state, Col 10, Lines 1-25 and 
generating request and other dialog prompts, Col 9, Lines 1-57; and output devices, Col 8, 
Lines 17-37); and 

A respective interface component associated with each peripheral device within said first 
and second sets; wherein each interface component is configured to provide the second data, if 
any, to the associated peripheral device and receive the first data, if any, from the associated 
peripheral device (I/O device drivers that provide input output data to their associated devices, 
Col 6, Lines 5-48; Col 7, Lines 1 7-27; and Col 8, Lines 1 7-3 7). 

With respect to Claim 2, Morin further discloses: 
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The control includes interrupting the prompt if the first data is received while the prompt 
is being output ("talk over" feature, Col 12, Lines 1-15). 
With respect to Claim 3, Morin further discloses: 

Delaying outputting of the prompt if one of the first set of peripheral devices is receiving 
the first data (dialog turns that require the dialog manager to wait for a complete user response 
before taking its turn, Col 9, Lines 35-56). 

With respect to Claim 7, Morin further discloses: 

The first set of peripheral devices includes one or more of a voice recognition system, a 
radio-frequency identifier scanner, a bar code scanner, a touch screen, a keypad, and a computer 
(input devices, Col 7, Lines 17-27). 

With respect to Claim 8, Morin further discloses: 

The second set of peripheral devices includes one or more of a voice synthesis system, a 
display screen and a computer (output devices, Col. 8, Lines 17-37). 
With respect to Claim 9, Morin discloses: 

Executing a workflow description received from the multimodal application, said 
workflow description including a plurality of workflow objects (application dialog script, which 
defines task operation sequences, executed by a dialog manager, Col 9, Line 1- Col 10, Line 
25; and Col. 14, Lines 1-23); 

Outputting a prompt of a first workflow object via a plurality of peripheral devices, said 
prompt related to the multimodal application (outputting task requests or messages to a user, Col. 
9, Lines 35-56; Col. 12, Lines 1-15); and 
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Controlling the outputting of the prompt according to an input state of the plurality of 
peripheral devices (dialog manager utilizing context and history to monitor an input state, Col 
10, Lines 1-25 and generating request and other dialog prompts, Col. 9, Lines 1-57; and output 
devices, Col. 8, Lines 17-37). 

With respect to Claim 11, Morin discloses the "talk over" feature, as applied to Claim 2. 

With respect to Claim 12, Morin further discloses: 

The step of outputting includes outputting an audio prompt (spoken messages, Col 12, 
Lines 1-15); and 

The step of receiving includes receiving voice data from a speech recognition system 
(speech recognizer, Col 7, Lines 28-48). 

With respect to Claim 13, Morin further discloses: 

The data is received from one of the plurality of peripheral devices (input devices, Col. 7, 
Lines 17-48). 

With respect to Claim 15, Morin further discloses: 

Performing the step of terminating if the data is received from a predetermined peripheral 
device; and omitting the step of terminating if the input is received from other than the 
predetermined device ("talk over" feature that interrupts a spoken prompt in response to a 
speech input and would inherently omit prompt cancellation if any other type of input was 
received, Col 12, Lines 1-15; and Col 7, Lines 28-32). 

Claim 19 contains subject matter similar to Claim 3, and thus, is rejected for the same 
reasons. 

With respect to Claim 20, Morin further discloses: 
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The step of delaying includes the steps of: delaying outputting the prompt to the one 
peripheral devices; and permitting outputting the prompt without delay to another of the plurality 
of peripheral devices (simultaneous output of text and speech and canceling only speech 
messages using a "talk-over" feature, Col. 12, Lines 1-15). 

Claim Rejections - 35 USC § 103 

10. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

1 1 . Claims 4-5 and 16-18 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Morin et al in view of Gergic et al (U.S. Patent Application Publication: 2002/0198719). 

With respect to Claims 4-5 and 16-17, Morin discloses the multimodal system and 
method featuring prompt control based on an input context and device drivers for processing 
input/output data, as applied to Claims 1 and 9. Although Morin discloses the ability to handle 
successive user responses (Col 12, Lines 41-54) and request a user response to fill-in missing 
information (Col 9, Lines 35-56), Morin does not specifically suggest processing a user input 
data directed to two prompts, which would result in an avoided subsequent prompt. Gergic, 
however, discloses the ability of a user to fill two fields of missing information with a single 
utterance in response to an initial dialog, which does not require the output of a subsequent 
dialog prompt (Paragraph 0121). 
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Morin and Gergic are analogous art because they are from a similar field of endeavor in 
interactive multimodal dialog systems. Thus, it would have been obvious to a person of ordinary 
skill in the art, at the time of invention, to modify the teachings of Morin with the ability of a 
user to fill two fields of missing information taught by Gergic in order to provide a means for 
supporting parallel activation of dialog objects (Gergic, Paragraph 0104) and allow for more 
efficient user/machine interaction. 

With respect to Claim 18, Gergic further recites: 

The data relates to the first workflow object and a plurality of other workflow objects 
(filling more than one field, Paragraph 0121). 

12. Claims 6 and 14 are rejected under 35 U.S.C. 103(a) as being unpatentable over Morin et 
al in view of Hanle et al (U.S. Patent: 5,012,511). 

With respect to Claims 6 and 14, Morin discloses the multimodal system and method 
featuring prompt control based on an input context and device drivers for processing input/output 
data, as applied to Claims 2 and 1 1 . Morin does not teach the use of non-interruptible prompts, 
however Hanle discloses the use of such prompts in an interactive dialog system (Col. 9, Lines 
33-51). 

Morin and Hanle are analogous art because they are from a similar field of endeavor in 
interactive voice response systems. Thus, it would have been obvious to a person of ordinary 
skill in the art, at the time of invention, to modify the teachings of Morin with the non- 
interruptible prompts taught by Hanle in order to force the user to listen to important prompts, 
such as error prompts (Hanle, Col. 9, Lines 50-51). 
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13. Claims 10, 22-23, and 25 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Morin et al in view of French-St. George et al (U.S. Patent: 6,012,030). 

With respect to Claims 10, Morin discloses the multimodal system and method featuring 
prompt control based on an input context and device drivers for processing input/output data, as 
applied to Claim 9. Although Morin teaches the concept of speech control of computer 
applications (Col. 13, Lines 61-67), Morin does not explicitly disclose prompts relating to visual 
control of a GUI screen, however French-St. George discloses such prompts (Col. 7, Line 37- 
Col 8, Line 18). 

Morin and French-St. George are analogous art because they are from a similar field of 
endeavor in interactive multimodal dialog systems. Thus, it would have been obvious to a 
person of ordinary skill in the art, at the time of invention, to modify the teachings of Morin with 
the generation of prompts relating to visual control of a GUI screen as taught by French-St. 
George in order to provide a means for dynamically adjusting audio prompts based on a user's 
interaction (French-St. George, Col. 3, Line 64- Col. 4, Line 2). 

Claim 22 contains subject matter similar to Claims 9 and 10, and thus, is rejected for the 
same reasons. Morin further discloses method implementation at a server computer (Col 5, 
Lines 43-56), which would inherently require some type of stored instruction set for method 
execution. 

With respect to Claim 23, Morin discloses the "talk over" feature, as applied to Claim 2. 
Claim 25 contains subject matter similar to Claim 3, and thus, is rejected for the same 
reasons. 
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14. Claim 21 is rejected under 35 U.S.C. 103(a) as being unpatentable over Morin et al in 
view of Setlur et al (U.S. Patent: 5,956,675). 

With respect to Claim 21, Morin discloses the multimodal system and method featuring 
prompt control based on an input context and device drivers for processing input/output data, as 
applied to Claim 19. Morin does not teach analyzing if a user's input relates to a prompt to 
identify a valid barge-in attempt, however, Setlur recites executing a barge-in attempt only when 
a speech input is related to an expected aural prompt response (Col 5, Lines 1-23). 

Morin and Setlur'are analogous art because they are from a similar field of endeavor in 
interactive voice response systems. Thus, it would have been obvious to a person of ordinary 
skill in the art, at the time of invention, to modify the teachings of Morin with the barge-in 
identification means taught by Setlur in order to provide a means for improved barge-in 
reliability (Setlur, Col. 5, Lines 18-20). 

15. Claim 24 is rejected under 35 U.S.C. 103(a) as being unpatentable over Morin et al in 
view of French-St George et al and further in view of Gergic et al (U.S. Patent Application 
Publication: 2002/0198719). 

With respect to Claim 24, Morin in view of French-St. George discloses the multimodal 
system and method featuring prompt control based on an input context and device drivers for 
processing input/output data, as applied to Claim 22. Although Morin discloses the ability to 
handle successive user responses (Col 12, Lines 41-54) and request a user response to fill-in 
missing information (Col 9, Lines 35-56), Morin does not specifically suggest processing a user 
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input data directed to two prompts, which would result in an avoided subsequent prompt. 
Gergic, however, discloses the ability of a user to fill two fields of missing information with a 
single utterance in response to an initial dialog, which does not require the output of a 
subsequent dialog prompt (Paragraph 0121). 

Morin, French-St. George, and Gergic are analogous art because they are from a similar 
field of endeavor in interactive multimodal dialog systems. Thus, it would have been obvious to 
a person of ordinary skill in the art, at the time of invention, to modify the teachings of Morin in 
view of French-St. George with the ability of a user to fill two fields of missing information 
taught by Gergic in order to provide a means for supporting parallel activation of dialog objects 
(Gergic, Paragraph 0104) and allow for more efficient user/machine interaction. 

Conclusion 

16. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure: 

Abella et al (U.S. Patent: 6,044,347)- discloses an object-oriented dialog manager. 

Marx et al (U.S. Patent: 6,173,266)- discloses an interactive dialog design system. 

Pickering (U.S. Patent: 6,496,799)- discloses a means for determining an end-of- 
utterance for issuing a voice prompt. 

Ferrans et al (U.S. Patent: 7,003,464)- teaches a dialog enabler for a voice browser. 

Guenther et al (U.S. Patent: 7,146,323)- teaches a multimodal web-based application 
form that can be filled based on user responses in a dialog. 
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Bers et al ("Designing Conversational Interfaces with Multimodal Interaction, " 1998)- 
discloses the use of an interactive dialog and prompt control in a GUI-based application. 

Wang- (SALT: A spoken Language Interface for Web-based Multimodal Dialog 
Systems, " 2002)- teaches a method for generating a prompt dialog from a GUI. 

17. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to James S. Wozniak whose telephone number is (571) 272-7632. 
The examiner can normally be reached on M-Th, 7:30-5:00, F, 7:30-4, Off Alternate Fridays. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Patrick Edouard can be reached at (571) 272-7603. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 

James S. Wozniak V. PAUL HARPER 

3/22/2007 PRIMARY PATENT EXAMINER 




